Outline of extreme test sensitivity analysis

The extreme conditions test is a stress test where input parameters are varied beyond probabilistic ranges, and the model-generated outcome behaviour is examined for consistency with the real or anticipated behaviour of the system. Models that exhibit logical consistency outside of ‘normal’ conditions instil greater confidence in the model output.

Benefits of the extreme conditions test:

• 1 It can be used to discover flaws in the model’s conceptual logic.
• 2 Aids in identifying non-linear and asymptotic behaviours that can be used to validate the model behaviour and highlight possible impactful policy levers.
• 3 Enhances the model’s usefulness by exploring model behaviour of systems that operate outside of contextual-historical observations, i.e., populations with differing characteristics.

Dipiction of sampled parameters

Model overview

There was a total of 70 model input values tested and 47 policy scenario input values tested, 117 variables in total. Most of these variables had age-gender-BMI specific values or related to food or physical activity subgroups, which were tested individually when appropriate. From this 117 variables, there was 746 subgroup input values (649 from the base model and 97 policy scenario inputs). A table describing variables being testes can be found in the appendix (Table 2)

Figure 1: Model overview

Base model parameters

Figure 2 shows the energy balance model component with the tested parameters circled in red. These parameters describe characteristics and definitions about the model. For example, the model development choices to aggregate discretionary foods according to the AUSNUT database impact the levels of consumption and the proportion of nutrients within each food group. If the food items included in the definitions of discretionary food are different from those chosen for the model, then these values would change. Numerical inputs are sampled to each of these circled variables individually, the impact of these different initial conditions then propagates throughout the model, impacting the outcomes. Many of these variables are stratified by age-gender-BMI groups.

Figure 2: Enegy balance component with circles representing sources of uncertainty

Figure 3 illustrates the structure that influences how children enter the model. The associations from the individual-level analysis of combined early prevention of obesity trials are varied to reflect the uncertainty associated with these regression coefficients.

Figure 3: Intergenerational structure parameters

Outcome Measure

Elementary Effect (EE) was used to summarise the model outcome. These summaries are employed to examine model behaviour. Variables highlighted as influential prompt the generation of individual input-output relationship plots, which disaggregate the summary results to further examine behaviours.

The Figure 6 below describes how Elementary Effect (EE) is used to summarise sensitivity analysis results.

Panel A shows an example of a model-generated outcome; over the modelled time, the input assumptions (\(x\)) are varied between \(\pm\) 20% of their initial value. The model results at the end of the model are used to calculate the EE (Panel B).

The EE is calculated for each model output result to create a sample of EE. The mean and standard deviation are used to summarise the change in outcome per percentage point shift in input assumption.

\[ EE_{i} = \frac{f(x-\Delta_{i})- f(x)}{\Delta_{i}}\]

\[ \mu_{EE}= \frac{1}{n} \sum_{i=1}^{n} EE_{i}, \hspace{1cm} \sigma_{EE}=\frac{1}{(n-1)} \sum_{i=1}^{n} (EE_{i} - \mu_{EE})^2\]

Figure 6: Example plot for analysis outcome

How to interpret the results

A summary plot (Panel A) displays the relationships between variation in the input variables (body weight of overweight males aged between 6 and 8) and the highlighted outcome, the prevalence of obesity.

The individual input-output relationships are plotted in the facets of Panel B. Each subplot shows the model-generated prevalence of obesity at the end of the model for each input assumption.

As the assumed body weight for overweight males aged between 6 and 8 is varied, the plots in Panel B show no impact on the prevalence of obesity for 2 to 5-year-olds, which is also reflected as a zero in the summary plot. The prevalence of obesity for 6 to 8-year-olds and above has a negative association with the assumed body weight of overweight males aged 6 to 8. The largest associations were in the 9 to 11-year-olds, and the strength of the association tapers out for older age groups.

For every percentage point change in the assumed body weight for overweight males aged 6 to 8 years old, there is, on average, a 0.25% reduction in the prevalence of obesity in 6 to 8-year-olds. This suggests that 25% of the input variability is reflected in the outcome.

Figure 7: Breakdown of results, changes in males body weight

Body Weight

Change in Male Body Weight input assumptions Changes in base model assumptions for body weight alter how changes in energy imbalance impact BMI category prevalence estimates, making the model insensitive to changes based on assumed energy surplus. The figure below shows the elementary effect for each local sensitivity analysis changing the assumed male body weight.

• Changes in the body weight assumptions reduced the flow of individuals to higher BMI categories, creating a higher prevalence of healthy weight.
• Younger ages impact the BMI outcomes for all older age groups.
• Older age groups do not impact younger outcomes.
• Changes in each BMI category mainly impact the outcome of the BMI category being changed and the neighboring BMI category.
• Changes in the assumptions for healthy weight impact the underweight, healthy weight, and overweight outcomes.
• Changes in the assumptions for overweight mainly impact the outcome for all BMI categories.
• Changing the obesity assumptions changes the obesity and overweight outcomes.

Change in Female Body Weight input assumptions

There are similar relationships, as noted in the male assumptions.

Additionally:

• The results show intergenerational effects.
• Changes in the female assumptions slightly impact the younger age group.
• The intergenerational effect is stronger when varying assumptions in age groups with higher fertility rates.
• For each percentage point (PP) increase in body weight for Underweight & Healthy weights 20 to 24 resulted in a:
  • 0.57% increase in Underweight & Healthy 20 to 24-year-olds.
  • 0.09% increase in Underweight & Healthy 2-year-old males (15.9% of the adults’ effect).
• For each PP increase in body weight for Underweight & Healthy weight 40 to 44 year-olds resulted in a:
  • 0.39% increase in Underweight & Healthy 40 to 44-year-olds.
  • 0.003% increase in Underweight & Healthy 2-year-old males (0.87% of the adults’ effect).

This suggests that any prevention intervention for early childhood should be targeted at age groups with higher fertility rates to have a larger impact.

Height

Similar to changes in the assumed body weight for each cohort, changes in height impact how influential energy surplus and deficits result in changes in the flows between BMI categories. However, height is the least influential in the model compared to body weight.

Change in Male height input assumptions

Change in Female height input assumptions

Schofield Equation - Intercept

Schofield Equation - Ceofficient

Growth Function kJ/day

Changes in the assumption of Kj/day needed for growth resulted in relatively small changes in the BMI outcomes. The largest impact occurred in adolescent age groups where the growth assumptions were the largest. The larger observed impact in males was in 9-11 year-olds and 12-15 year-olds, where on average, 1 pp resulted in a 0.2% change in the outcome.

Macronutrient energy density

Each food group is broken down into macronutrients; carbohydrates, protein, fats, and sugars. The energy from a gram of macronutrients is an assumed model input. This assumption impacts each food group for each age-gender-BMI group.

• The input of macronutrients becomes more sensitive for the older age group. This is due to the cumulative impact of the population’s life course.
• Higher kJ/g leads to higher daily total dietary intake, leading to a higher prevalence of overweight and obesity.
• For every percentage point change in the assumed energy density for dietary fats, there was, on average, a 1.08% increase in the prevalence of obesity in 12-14-year-olds for females and a 1.15% increase for males.
• This effect was 1.98% and 2.72% for 40-44-year-olds females and males, respectively.

Explanation of summary results

Thermic effect of food (TEF)

The thermic effect of food (TEF) is the proportion of the energy used for digestion. A higher TEF means less dietary energy is available after digestion. TEF input assumptions impact the whole population, translating to a highly sensitive input assumption.

• Change in TEF, cumulative over the life course leading to a higher impact for older age groups.
• A 1 percentage point (0.0015 TEF units) increase in the TEF of carbohydrate results in a -0.06 to -0.52% change in obesity.
• A -0.44 to -3.51% change in obesity per 0.01 TEF.
• A 1 percentage point (0.0015 TEF units) increase in the TEF of fat results in a -0.07 to -0.59% change in obesity.
• A -0.48 to -3.99% change in obesity per 0.01 TEF.
• A 1 percentage point (0.002 TEF units) increase in the TEF of protein results in a -0.06 to -0.49% change in obesity.
• A -0.30 to -2.45% change in obesity per 0.01 TEF.
• A 1 percentage point (0.0007 TEF units) increase in the TEF of sugar results in a -0.03 to -0.27% change in obesity.
• A -0.43 to -3.90% change in obesity per 0.01 TEF.

Explanation of summary results

Adult to Adult Social Transmission

Adult to Adult social transmission assumptions impact how changes in physical activity (PAL) and dietary behaviours are transmitted to other adults through role modelling. This assumption has little impact on the output. However, because this variable is dependent on change of behaviour, role-modelling likely interacts with intervention effects.

Adult to Child Social Transmission

Similar to “Adult to Adult” role-modelling, BMI outcome has little impact when Adult to Child role-modelling assumptions are varied.

Child to Child Social Transmission

BMI outcome has little impact when Child to Child role-modelling assumptions are varied. These assumptions interact with intervention assumptions.

Infant Reported behaviours

The assumed reported infant behaviours amplify the intergenerational effects. However, these impacts are minimal.

• The proportion of mothers that breastfeed impacts older age groups. This is because of the increase in energy expenditure caused by breastfeeding.
• The proportion of infants that consume non-core foods and TV for more than 1 hr/day impacts younger age groups. Higher non-core food consumption and greater TV viewing increase the prevalence of overweight and obesity.

Assumed intergenerational relationships

Mortality ratios

Hazard ratios are applied to exogenous mortality rate so predicted population dynamics are maintained.

• Changing these assumptions impact older age group, primarily cause by higher mortality rates in these age groups.
• Since the primary outcome is a percentage, these ratio have little impact.

METs

Duration and intensity (METs) of physical activity are used to estimate total energy expenditure. The assumed METs for each movement category are applied to all of the population, which results in highlighted sensitivity input assumptions.

• Impact from variations in METs assumptions accumulates over the population’s life course, resulting in higher impacts for older age groups.
• An increase in MET implies that each movement activity has higher intensity leading to a healthier population.
• A 1 percentage point increase in sleep METs results in a between -0.15% to -2.83% change in obesity.
• A 1 percentage point increase in inactive METs results in a between -0.14% to -2.81% change in obesity.
• A 1 percentage point increase in screen time METs results in a between -0.11% to -2.36% change in obesity.
• A 1 percentage point increase in light physical activity METs results in a between -0.08% to -1.63% change in obesity.
• A 1 percentage point increase in moderate physical activity METs results in a between -0.08% to -0.99% change in obesity.
• A 1 percentage point increase in vigorous physical activity METs results in a between 1.88% to 29.19% change in obesity.

Explanation of summary results

Light physical activity

Each behaviour is structured so that they are age-dependent; observed surveyed behaviours were modeled using linear regression with age groups as the independent variable. This creates two variables for each behaviour; the intercept is the level of behaviour at the youngest age group (2 years old), and the change of behaviours over the life course.

• Changes in the intercept assumptions have cumulative impacts over the life course, resulting in higher impacts in older age groups compared to younger age groups.
• Varying age-on-age (AGE SLOPE) changes the trajectory of behaviours over the life course, making older age group BMI outcomes sensitive to changes in age slope.

Moderate physical activity

Vigorous physical activity

Screen time

Sleep

Daily intake of Fruit

Daily intake of Vegetables

Daily intake of Grains

Daily intake of Dairy

Daily intake of Meat and Protein

Daily intake of Discretionary foods

Daily intake of Fats and Oils

Daily intake of Sugar-sweetened beverages

Daily intake of Water

Daily intake of Other beverages (non-water, non-ssb)

Daily intake of Other foods

Fat-mass (%)

Change in Male fat-mass % assumptions

Change in Female fat-mass % assumptions

Proportion of nutrients within food group Inputs

Grains

Vegetables

Fruit

Dairy

Meat and Protein

Fats and oils

Discretionary foods

Sugar-sweetened beverages

Other foods

Other beverages (non-ssb, non-water)

Initial BMI Prevalence Inputs

Change in Male Initial BMI Prevalence

Change in Female Initial BMI Prevalence # Conclusion The purpose of this analysis was to examine the results of an extreme input conditions sensitivity analysis to discover flaws in the model’s conceptual logic, understand model behaviour, and highlight possible impactful policy levers.

When changes were made to input assumptions, the changes in the outcome give insight into how the model reacts. In the results presented in this report, these changes acted in line with a purposefully built structure. The results also highlight that variables that globally define model characteristics, such as METs, TEF, energy density, and proportion of nutrients within food groups, can have large impacts on the modeled BMI prevalence.

Overall Ranking of sensitivity

The 649 input assumptions tested can be summarised into 70 variables by taking the mean of age-gender-BMI specific inputs, resulting in 70 overarching variables. These are ranked in the following table.

Table of variables and number of sensitivity runs

## R version 4.3.2 (2023-10-31)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Pop!_OS 20.04 LTS
## 
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 
## LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
## 
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
## 
## time zone: Australia/Sydney
## tzcode source: system (glibc)
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] scales_1.2.1     ggpubr_0.6.0     ggh4x_0.2.5      patchwork_1.1.3 
##  [5] openxlsx_4.2.5.2 lubridate_1.9.2  forcats_1.0.0    stringr_1.5.0   
##  [9] dplyr_1.1.2      purrr_1.0.2      readr_2.1.4      tidyr_1.3.0     
## [13] tibble_3.2.1     ggplot2_3.4.3    tidyverse_2.0.0  progress_1.2.2  
## 
## loaded via a namespace (and not attached):
##  [1] gtable_0.3.4      xfun_0.40         bslib_0.5.1       rstatix_0.7.2    
##  [5] lattice_0.22-5    tzdb_0.4.0        vctrs_0.6.3       tools_4.3.2      
##  [9] generics_0.1.3    fansi_1.0.4       highr_0.10        pkgconfig_2.0.3  
## [13] Matrix_1.6-1      webshot_0.5.5     lifecycle_1.0.3   compiler_4.3.2   
## [17] farver_2.1.1      munsell_0.5.0     carData_3.0-5     htmltools_0.5.6  
## [21] sass_0.4.7        yaml_2.3.7        pillar_1.9.0      car_3.1-2        
## [25] crayon_1.5.2      jquerylib_0.1.4   ggbrace_0.1.0     cachem_1.0.8     
## [29] abind_1.4-5       nlme_3.1-163      rvest_1.0.3       tidyselect_1.2.0 
## [33] zip_2.3.0         digest_0.6.33     stringi_1.7.12    bookdown_0.35.1  
## [37] labeling_0.4.2    rmdformats_1.0.4  splines_4.3.2     fastmap_1.1.1    
## [41] grid_4.3.2        colorspace_2.1-0  cli_3.6.1         magrittr_2.0.3   
## [45] utf8_1.2.3        broom_1.0.5       withr_2.5.0       prettyunits_1.1.1
## [49] backports_1.4.1   timechange_0.2.0  httr_1.4.7        rmarkdown_2.24   
## [53] ggsignif_0.6.4    hms_1.1.3         kableExtra_1.3.4  evaluate_0.21    
## [57] knitr_1.44        viridisLite_0.4.2 mgcv_1.9-0        rlang_1.1.1      
## [61] Rcpp_1.0.11       glue_1.6.2        xml2_1.3.5        svglite_2.1.1    
## [65] rstudioapi_0.15.0 jsonlite_1.8.7    R6_2.5.1          systemfonts_1.0.4